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Abstract 

We propose deterministic timed automata (DTA) as a model- 
independent language for specifying performance and dependability mea- 
sures over continuous-time stochastic processes. Technically, these mea- 
sures are defined as limit frequencies of locations (control states) of a DTA 
that observes computations of a given stochastic process. Then, we study 
the properties of DTA measures over semi-Markov processes in greater 
detail. We show that DTA measures over semi-Markov processes are well- 
defined with probability one, and there are only finitely many values that 
can be assumed by these measures with positive probability. We also give 
an algorithm which approximates these values and the associated probabil- 
ities up to an arbitrarily small given precision. Thus, we obtain a general 
and effective framework for analysing DTA measures over semi-Markov 
processes. 

1 Introduction 



Continuous-time stochastic processes, such as continuous-time Markov chains, 
semi-Markov processes, or generalized semi-Markov processes [221 El 1201 IUji 
have been widely used in practice to determine performance and dependability 
characteristics of real-world systems. The desired behaviour of such systems is 
specified by various measures such as mean response time, throughput, expected 
frequency of errors, etc. These measures are often formulated just semi-formally 
and chosen specifically for the system under study in a somewhat ad hoc man- 
ner. One example of a rigorous and mo del- independent specification language for 
performance and dependability properties is Continuous Stochastic Logic (CSL) 
[21 [5] which allows to specify both steady state and transient measures over the 
underlying stochastic process. The syntax and semantics of CSL is inspired by 



the well-known non-probabilistic logic CTL [T3]. The syntax of CSL defines state 
and path formulae, interpreted over the states and runs of a given stochastic pro- 
cess M. In particular, there are two probabilistic operators, V^g{-) and 4S>^g(-), 
which refer to the transient and steady state behaviour of M , respectively. Here 
ixi is a numerical comparison (such as <) and g £ [0, 1] is a rational constant. If 
iy9 is a path formulc0 (which is either valid or invalid for every run of A4), then 
'P>Q.7{'p) is a state formula which says "the probability of all runs satisfying 
(f is at least 0.7". If $ is a state formula, i.e., $ is either valid or invalid in 
every state, then iS>o.5(3>) is also a state formula which says "the 7r-weighted 
sum over all states where $ holds is at least 0.5". Here tt is the steady-state 
distribution of Ai . The logic CSL can express quite complicated properties and 
the corresponding mo del- checking problem over continuous-time Markov chains 
is decidable. However, there are also several disadvantages. 

(a) The semantics of steady state probabilistic operator iSxig(-) assumes the 
existence of invariant distribution which is not guaranteed to exist for 
all types of stochastic processes with continuous time (the existing works 
mainly consider CSL as a specification language for ergodic continuous- 
time Markov chains). 

(b) In CSL formulae, all measures are explicitly quantified, and the model- 
checking algorithm just verifies constraints over these measures. Alterna- 
tively, we might wish to compute certain measures up to a given precision. 

In this paper, we propose deterministic timed automata (DTA j as a model- 
independent specification language for performance and dependability measures 
of continuous-time stochastic processes. The "language" of DTA can be inter- 
preted over arbitrary stochastic processes that generate timed words, and their 
expressive power appears sufficiently rich to capture many interesting run-time 
properties (although we do not relate the expressiveness of CSL and DTA for- 
mally, they are surely incomparable because of different "nature" of the two 
formalisms). Roughly speaking, a DTA A "observes" runs of a given stochas- 
tic process M. and "remembers" certain information in its control states (which 
are called locations). Since A is deterministic, for every run a oi Ai there is a 
unique computation A(a) of A, which determines a unique tuple of "frequen- 
cies" of visits to the individual locations of A along a. These frequencies are the 
values of "performance measures" defined by A (in fact, we consider discrete and 
timed frequencies which are based on the same concept but defined somewhat 
differently). 

Let us explain the idea in more detail. Consider some stochastic process M 
whose computations (or runs) are infinite sequences of the form cr = sq si • • ■ 
where all Si are "states" and ti is the time spent by performing the transition from 
Si to Si+i. Also assume a suitable probability space defined over the runs of M.. 
Let S by a finite alphabet and L a labelling which assigns a unique letter L[s) S S 
to every state s of A^. Intuitively, the letters of E correspond to collections of 

^In CSL, (/3 can be of the form Xj'^ or <E>iW/"I>2 where <E>,<l?i,<3'2 are state formulae, and 
Xj,Uj are the modal connectives of CTL parametrized by an interval /. Boolean connectives 
can be used to combine just state formulae. 
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predicates that are valid in a given state. Thus, every run cr = Sq • • • of 

M. determines a unique timed word w^- = L{so) to L{s\) ti - ■ ■ over S. 

A DTA over S is a finite-state automaton A equipped with finitely many 
internal clocks. Each control state (or location) g of ^ has finitely many out- 
going edges q — >q' labeled by triples {a,g,X), where a G T,, g is a. "guard" 
(a constraint on the current clock values), and X is a subset of clocks that are 
reset to zero after performing the edge. A configuration of .4. is a pair (g, z^), 
where q and u are the current location and the current clock valuation, respec- 
tively. Every timed word w = cq Ci C2 C3 ■ ■ ■ over S (where G S iff i is even) 
then determines a unique run A{w) = {qo, I'o) {qi, J^i) (92, 1^2) • • • of ^ where go 
is an initial location, vq assigns zero to every clock, and {qi^i,Ui^i) is obtained 
from {qi,Vi) either by performing the only enabled edge qi — > qi+i labeled by 
{ci,g,X) if i is even, or by simultaneously increasing all clocks by Cj if i is odd. 

As a simple example, consider the following DTA A over the alphabet {a} 
with one clock x and the initial location qQ-. 

a, X < 2, x:=0 

' ■ ■ <2, x:=0 




a, a: > 2, x:— 

X > 2, x:=0 



a, X > 2, X 

Intuitively, A observes time stamps in a given timed word and enters either q\ 
or q], depending on whether a given stamp is bounded by 2 or not, respectively. 
For example, a word w = a 0.2 a 2.4 a 2.1 •• • determines the run A{'w) = 
{qo, 0) (gi, 0) (gi, 0.2) (gt, 0) (gt, 2.4) {qi, 0) {qi, 2.1) • • • 

Let w = aotottiti ■ ■ ■ be a timed word over E and q a location of A. For 
every i G No, let T'^{w) be the stamp ti of w, and Q^{w) the location of A entered 
after reading the finite prefix aoto - ■ - Ui of w. Further, let lg{w) be either 1 or 
depending on whether Q'^{w) = q ov not, respectively. We define the discrete 
and timed frequency of visits to q along A{w), denoted by d^(?«) and C'^{w), in 
the following way (the '.4,' index is omitted when it is clear from the context): 



•^g^(^) = limsup 
*^g^(^) = limsup 



n 

Y.UT\w)-Vg{w) 



Thus, every timed word w determines the tuple d-^(w) = (d^(w))^^^ and the 
tuple c-^(m)) = of discrete and timed A-measures, respectively. 

DTA measures can encode various performance and dependability properties 
of stochastic systems with continuous time. For example, consider again the 
DTA A above and assume that all states of a given stochastic process M. are 
labeled with a. Then, the fraction 

Aq\{Wa) 



dgt(Wa) + ^qi{Wa) 
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corresponds to the percentage of transitions of that are performed within 2 
seconds along a run a. If is an ergodic continuous-time Markov chain, then 
the above fraction takes the same value for almost all runs ct of A^. However, 
it makes sense to consider this fraction also for non-ergodic processes. For ex- 
ample, we may be interested in the expected value of dg-f/(dg-|- -I- d,^), or in the 
probability of all runs a such that the fraction is at least 0.5. 

One general trouble with DTA measures is that (i^{w) and c^(w) faithfully 
capture the frequency of visits to q along w only if the limits 

exist, in which case we say that d-^ and are well-defined for w, respectively. 
So, one general question that should be answered when analyzing the properties 
of DTA measures over a particular class of stochastic processes is whether d-^ 
and are well-defined for almost all runs. If the answer is negative, we might 
either try to re-design our DTA or accept the fact that the limit frequency of 
the considered event simply does not exist (and stick to limsup). 

In this paper, we study DTA measures over semi-Markov processes (SMPs). 
An SMP is essentially a discrete-time Markov chain where each transition is 
assigned (apart of its discrete probability) a delay density, which defines the 
distribution of time needed to perform the transition. A computation (run) of 
an SMP At is initiated in some state sq, which is also chosen randomly according 
to a fixed initial distribution over the state space of A4 . The next transition is 
selected according to the fixed transition probabilities, and the selected transition 
takes time chosen randomly according to the density associated to the transition. 
Hence, each run of M is an infinite sequence sqIq siti ■ ■ ■ , where all Si are states 
of M and ti are time stamps. The probability of (certain) subsets of runs in M. 
is measured in the standard way (see Section 

The main contribution of this paper are general results about DTA measures 
over semi-Markov processes, which are valid for all SMPs where the employed 
density functions are bounded from zero on every closed subinterval (see Sec- 
tion [2]). Under this assumption, we prove that for every SMP A4 and every DTA 
A we have the following: 

(1) Both discrete and timed y^- measures are well defined for almost all runs of 
M. 

(2) Almost all runs of A4 can be divided into finitely many pairwise disjoint 
subsets TZi,...,TZk so that d-^{w) takes the same value for almost all 
w (iTZj, where 1 < j < fc. The same result holds also for c-^. (Let us 
note that k can be larger than 1 even if Ai is strongly connected.) 

(3) The observations behind the results of (1) and (2) can be used to compute 
the k and effectively approximate the probability of all TZj together with 
the associated values of discrete or timed ^-measures up to an arbitrarily 
small given precision. More precisely, we show that these quantities are 
expressible using the m-step transition kernel P™ of the product process 
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M X A defined for M and A (see Sectfon lX^ . and we give generic bounds 
on the number of steps m that is sufficient to achieve the required precision. 
The m-step transition kernel is defined by nested integrals (see Section [571]) 
and can be approximated by numerical methods (see, e.g., [16l|9]). This 
makes the whole framework effective. The design of more efficient algo- 
rithms as well as more detailed analysis applicable to concrete subclasses 
of SMP are left for future work. 

To get some intuition about potential applicability of our results (and about 
the actual power of DTA which is hidden mainly in their ability to accumulate 
the total time of several transitions in internal clocks), let us start with a sim- 
ple example. Consider the following itinerary for travelling between Brno and 
Prague: 

Brno Kufim Tisnov Caslav Prague 

arrival 1:15 2:30 3:30 4:50 

departure 0:00 1:20 2:40 3:35 

A traveller has to change a train at each of the three intermediate stops, and she 
needs at least 3 minutes to walk between the platforms. Assume that all trains 
depart on time, but can be delayed. Further, assume that travelling time between 
X and Y has density fx- y- We wonder what is the chance that a traveller reaches 
Prague from Brno without missing any train and at most 5 minutes after the 
scheduled arrival. Answering this question "by hand" is not simple (though still 
possible). However, it is almost trivial to rephrase this question in terms of DTA 
measures. The itinerary can be modeled by the following semi-Markov process, 
where the density / is irrelevant and S = {B,K,T,C,P}. 

^-N Ib-K ^— n fK-T^^--\ fr-c fc-p ^-N 

/ 

The property of "reaching Prague from Brno without missing any train and at 
most 5 minutes after the scheduled arrival" is encoded by the DTA A of Figure[TJ 
The automaton uses just one clock x to measure the total elapsed time, and the 
guards reflect the required timing constraints. Starting in location init^ the 
automaton eventually reaches either the location or pi, which corresponds to 
satisfaction or violation of the above property, and then it is "restarted" . Hence, 
we are interested in the relative frequency of visits to p'\ among the visits to p'l 
or p^. Using our results, it follows that d-^ is well-defined and takes the same 
value for almost all runs of A4. Hence, the random variable dp-|-/(dp-t- -I- dpj,) 
also takes the same value with probability one, and this (unique) value is the 
quantity of our interest. 

Now imagine we wish to model and analyse the flow of passengers in London 
metro at rush hours. The SMP states then correspond to stations, transition 
probabilities encode the percentage of passengers traveling in a given direction, 
and the densities encode the distribution of travelling time. A DTA can be used 
to monitor a complex list of timing restrictions such as "there is enough time 
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to change a train" , "travelling between important stations does not take more 
than 30 minutes if one the given routes is used" , "trains do not arrive more than 
2 minutes later than scheduled" , etc. For this we already need several internal 
clocks. Apart of some auxiliary locations, the constructed DTA would also have 
special locations used to encode satisfaction/ violation of a given restriction (in 
the DTA A of Figure [H (p, t) and {p,i) are such special locations). Using the 
results presented in this paper, one may not only study the overall satisfaction 
of these restrictions, but also estimate the impact of changes in the underlying 
model (for example, if a given line becomes slower due to some repairs, one may 
evaluate the decrease in various dependability measures without changing the 
constructed DTA). 

Proof techniques. For a given SMP A4 and a given DTA A we first construct 
their synchronized product Ai x A, which is another stochastic process. In fact, 
it turns out that x ^ is a discrete-time Markov chain with uncountable state- 
space. Then, we apply a variant of the standard region construction [2] and thus 
partition the state-space oi Ai x A into finitely many equivalence classes. At 
the very core of our paper there are several non-trivial observations about the 
structure of x ^ and its region graph which establish a powerful link to the 
well-developed ergodic theory of Markov chains with general state-space (see, 
e.g., [mUl]). In this way, we obtain the results of items (1) and (2) mentioned 
above. Some additional work is required to analyze the algorithm presented in 
Section |4] (whose properties are summarized in item (3) above). 

Related work. There is a vast literature on continuous-time Markov chains, 
semi-Markov processes, or even more general stochastic models such as gener- 
ahzed semi-Markov processes (we refer to, e.g., [23l |6l [20l [IT]). In the com- 
puter science context, most works on continuous-time stochastic models concern 
model-checking against a given class of temporal properties [3l [5] . The usefulness 
of CSL model-checking for dependability analysis is advocated in [M]. Timed 
automata [5] have been originally used as a model of (non-stochastic) real-time 
systems. Probabilistic semantics of timed automata is proposed in [4l[7]. The 
idea of using timed automata as a specification language for continuous-time 
stochastic processes is relatively recent. In [12], the model-checking problem 
for continuous-time Markov chains and linear-time properties represented by 
timed automata is considered (the task is to dermine the probability of all timed 
words that are accepted by a given timed automaton). A more general model of 
two-player games over generalized semi-Markov processes with qualitative reach- 
ability objectives specified by deterministic timed automata is studied in [lOj . 

2 Preliminaries 

In this paper, the sets of all positive integers, non-negative integers, real numbers, 
positive real numbers, and non- negative real numbers are denoted by N, No, M, 
R>o, and M>o, respectively. 

Let A be a finite or countably infinite set. A discrete probability distribution 
on ^ is a function a : A ]R>o such that J^aeA'^i'^) ~ ^- ^^^^ '^^ 

rational if a{a) is rational for every a ^ A. The set of all distributions on A 
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Figure 1: A deterministic timed automaton A. 

is denoted by 'D{A). A a -field over a set is a set C 2^ that includes VL 
and is closed under complement and countable union. A measurable space is a 
pair (ri, J-) where $7 is a set called sample space and 7^ is a cr-field over Q, whose 
elements are called measurable sets. A probability measure over a measurable 
space {^tJ^) is a function V : J- ^ M>o such that, for each countable collec- 
tion {Xi\i^i of pairwise disjoint elements of 'P(Uie/ -^i) ~ X^ig/ "Pi^i): and 
moreover 'P{fl) = 1. A probability space is a triple {Q,J^,V), where is 
a measurable space and P is a probability measure over ($7, J^). We say that a 
property A C holds for almost all elements of a measurable set Y if ViY) > 0, 

^nr e J", and V{A I y) = 1. 

All of the integrals used in this paper should be understood as Lebesgue 
integrals, although we use Riemann-like notation when appropriate. 

2.1 Semi-Markov processes 

A semi-Markov process (see, e.g., [23]) can be seen as discrete-time Markov 
chains where each transition is equipped with a density function specifying the 
distribution of time needed to perform the transition. Formally, let S be a set of 
delay densities, i.e., measurable functions f -.M. ^ R>o satisfying f{t) dt — 1 
where f{t) = for every t < 0. Moreover, for technical reasons, we assume that 
each f ^ 1) satisfies the following: There is an interval / either of the form [£, u] 
with £,u £ No, ^ < M, or [£, oo) with € No, such that 

• for alH e M \ / we have that f{t) = 0, 

• for all [c, d] C I there is 6 > such that for all t G [c, d] we have that 
fit) > b. 

The assumption that £, u are natural numbers is adopted only for the sake of 
simplicity. Our results can easily be generalized to the setting where / is an 
interval with rational bounds or even a finite union of such intervals. 

Definition 2.1. A semi-Markov process (SMP) is a tuple Ai = (5, P,D,ao), 

where S is a finite set of states, P : — > I?(S') is a transition probability function, 
D : S X S ^ 'D is a delay function which to each transition assigns its delay 
density, and ao G 2?(S') is an initial distribution. 
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A computation (run) of a SMP ^4 is initiated in some state Sq, which is 
chosen randomly according to ao- In the current state Si, the next state s^+i is 
selected randomly according to the distribution P(si), and the selected transition 
(si,Si+i) takes a random time U chosen according to the density D(si,Si+i). 
Hence, each run of A4 is an infinite timed word sq to si ti ■ ■ • , where Si € S and 
ti e ]R>o for all z S Nq. We use TZm to denote the set of all runs of M. 

Now we define a probability space {TZmjJ^Mj'Pm) over the runs of A4 (we 
often omit the index M if it is clear from the context). A template is a finite 
sequence of the form B = sq Iq si h ■ ■ ■ s„+i such that n > and li is an interval 
in R>o for every < i < n. Each such B determines the corresponding cylinder 
Ti{B) C TZ consisting of all runs of the form Sq to Si ti ■ ■ • , where Si — Si for 
all < i < and ti e I., for all < i < n. The cr-field F is the Borel 

a-field generated by all cylinders. For each template B = sq lo si Ii ■ ■ ■ Sn+i, 
let Pi — F{si){si+i) and fi — D(si,Si+i) for all < i < n. The probability 
V{TZ{B)) is defined as follows: 



Then, V is extended to J- (in the unique way) by applying the extension theorem 
(see, e.g., [8]). 

2.2 Deterministic timed automata 

Let X he a finite set of clocks. A valuation is a function : X ^ K>o- For 
every valuation and every subset X C X oi clocks, we use iy[X := 0] to 
denote the unique valuation such that i^lX :— 0]{x) is equal either to or ^{x), 
depending on whether x Cz X or not, respectively. Further, for every valuation 
u and every S G M>o, the symbol v + S denotes the unique valuation such that 
{u + 6){x) — J^(x) + 6 for all x € X. Sometimes we assume an implicite linear 
ordering on clocks and slightly abuse our notation by identifying a valuation v 
with the associated vector of reals. 

A clock constraint (or guard) is a finite conjunction of basic constraints of 
the form x [xi c, where a; G A", ixi G {<, <, >, >}, and c G Nq. For every valuation 
v and every clock constraint g we have that ly either does or does not satisfy g, 
written v \= g ot v ^ g, respectively (the satisfaction relation is defined in the 
expected way). Sometimes we identify a guard g with the set of all valuations 
that satisfy g and write, e.g., g D g'. The set of all guards over X is denoted 



Definition 2.2. A deterministic timed automaton (DTA) is a tuple A = 
{Q,T.,X, — >,qo), where Q is a nonempty finite set of locations, T, is a fi- 
nite alphabet, X is a finite set of clocks, qo € Q is an initial location, and 
— > C Q X S X B{X) X 2'^ X Q is an edge relation such that for all q & Q and 
a e S we have the following: 

1. the guards are deterministic, i.e., for all edges of the form (q, a, gi, Xi, qi) 
and {q, a, g2, X2, q2) such that gi Cl g2 ^ ^ we have that gi — g2, A"i — X2, 
and qi = 52/ 




n 



by B{X). 



8 



2. the guards are total, i.e., for all q Q, a E S, and every valuation v there 
is an edge [q, a, g, X, q') such that v \^ g. 

A configuration of ^ is a pair (g, v), where q (z Q and v is a valuation. An infinite 
timed word over E is an infinite sequence w = cq ci 02 03 ■ ■ ■ , wliere Ci £ S when i 
is even, and Ci G ]R>o when i is odd. The run of ^ on w is the unique infinite 
sequence of configurations A{w) — {qo, vq) vi) ■ ■ ■ such that go is the initial 
location of A, vo{x) = for all a; e A", and for each i £ No we have that 

• if Ci is a time stamp, then qi+i = qi and Vi+i ^ i>i + Ci\ 

• if Ci is a letter of S, then there is a unique edge {qi,Ci,g,X,q) such that 
Vi \= g, and we require that g^+i — q and m^+i — VilX := 0]. 

Notice that we do not define any acceptance condition for DTA. Instead, we 
understand DTA as finite-state observers that analyze timed words and report 
about certain events by entering designated locations. The "frequency" of these 
events is formally captured by the quantities dq and Cq defined below. 

Let A = {Q, E, X, — >, qo) be a DTA, q E Q some location, and w = 
aotoQiti ■ ■ ■ a timed word over E. For every i G No, let T^{w) be the stamp ti 
of w, and Q^{w) the unique location of A entered after reading the finite pre- 
fix ao io • • • OLi of w. Further, let Vq(w) be either 1 or depending on whether 
Q^{w) = q or not, respectively. The discrete and timed frequency of visits to q 
along A{w), denoted by d^(w) and c^{w), are defined in the following way (if 
A is clear, it is omitted): 

cf{w) 

Hence, every timed word w determines the tuple d-^ = (d^(it;))^^^ and the 
tuple — (c^(ii;))^^^ of discrete and timed A-measures, respectively. The A- 
measures were defined using limsup, because the corresponding limits may not 
exist in general. If lim„_j.oo SILi ^qi"^)/^ exists for all g S Q, we say that d'^ 
is well-defined for w. Similarly, if lim„_j.oo(X)r=i ^*(^) ' lg(^))/(Sr=i ^*(^)) 
exists for all q, we say that is well-defined for w. 

As we already noted in Section [U a DTA A can be used to observe runs 
in a given SMP Ai after labeling all states of A4 with the letters of E by a 
suitable L : 5 ^ E. Then, every run a = soto siti ■ ■ ■ oi A4 determines a unique 
timed word Wa = L(so) to L{si) ti ■ ■ ■ , and one can easily show that for every 
timed word w, the set {a & TZ \ Wa = w} is measurable in {TZ, T, V). 

3 DTA Measures over SMPs 

Throughout this section we fix an SMP M = (S*, P,D,ao) and a DTA A = 
(Q, E, A", — qo) where X = {xi, . . . , To simplify our notation, we assume 



= lim sup 



= lim sup 
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that 11 = 3, i.e., every run tr of is a timed word over S (hence, we do not 
need to introduce any labehng L : — > S). This technical assumption does not 
affect the generality of our results (all of our arguments and proofs work exactly 
as they are, we only need to rewrite them using less readable notation). Our 
goal is to prove the following: 

Theorem 3.1. 

1. d-^ is well-defined for almost all runs of M.. 

2. There are pairwise disjoint sets TZi , . . . , TZk of runs in M such that V{TZi U 
• • -LlTZk) = 1, and for every 1 < j < k there is a tuple Dj such that d-^{a) = 
Dj for almost all a G TZj (we use Dj q to denote the q-component of Dj). 

In Section m we show how to compute the k and approximate V(TZj) and Dj 
up to an arbitrarily small given precision. 

An immediate corollary of Theorem 13.11 is an analogous result for c"^ . 

Corollary 3.2. c-^ is well-defined for almost all runs of M. Further, there are 
pairwise disjoint sets TZi, . . . , TZk of runs in A4 such that V{Tii U • • • U7?.a') — 1, 
and for every 1 < j < K there is a tuple Cj such that c-^(cr) = Cj for almost all 
a €71 J. 

Corollary 13.21 follows from Theorem 13.11 simply by considering the discrete 
jjSx^ measure, where the DTA 5xy^ is obtained from A in the following way: the 
set of locations of x ^ is {^o} U (S" x Q), and for every transition {qo, s, g, X, q') 
of A we add a transition {qQ,s,g,X, {s,q')) to S" x and for every transition 
{q, s, g, X,q') and every s' € S we add a transition {{s' ,q), s, g, X, {s,q')) to 
S X A. The initial location of S" x ^ is go- Intuitively, S* x ^ is the same as 
A but it explicitly "remembers" the letter which was used to enter the current 
location. Let k and Dj be the constants of Theorem 13.11 constructed for A4 and 
S* X Observe that the expected time of performing a transition from a given 
s G S, denoted by Eg, is given by Eg = X^s'eS ■^(*)(^') ' ^s,s', where E^^s' is the 
expectation of a random variable with the density D(s, s'). From this we easily 
obtain that 

for all g G Q and 1 < j < k. The details are given in Appendix [X] Hence, 
we can also compute the constant K and approximate V{TZj) and Cj for every 
^ < j ^ K using Equation ^ . 

It remains to prove Theorem 13.11 Let us start by sketching the overall 
structure of our proof. First, we construct a synchronous product M x A 
of A4 and A, which is a Markov chain with an uncountable state space 
^MxA = S X Q X (M>o)". Intuitively, M x A behaves in the same way as M 
and simulates the computation of A on-the-fly (see Figure[2]) . Then, we construct 
a finite region graph GmxA over the product Ai x A. The nodes of Gm X jA are 
the sets of states that, roughly speaking, satisfy the same guards of A. Edges are 
induced by transitions of the product (note that if two states satisfy the same 
guards, the sets of enabled outgoing transitions are the same). By relying on 
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M : (£o) — ^(JD — ^(5) 




MxA: 





to 




tl 




1 so, 90, 1^0 
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Figure 2: Synchronizing and ^ in x ^. Notice that vq = i^q — and 

t^i+l = t'li + ii- 

arguments presented in [1] [10] , we show that ahiiost all runs reach a node of a 
bottom strongly connected component (BSCC) C of GmxA (by definition, each 
run which enters C remains in C). This gives us the partition of the set of runs 
of A4 into the sets TZi , ■ ■ ■ , TZk (each TZj corresponds to one of the BSCCs of 
Gmxa)- 

Subsequently, we concentrate on a fixed BSCC C, and prove that almost all 
runs that reach C have the same frequency of visits to a given q (z Q (this gives 
us the constant Dj ^). Here we employ several deep results from the theory of 
general state space Markov chains (see Theorem l3.6|) . To apply these results, we 
prove that assuming aperiodicity of GmxA (see Definition l3.10p . the state space 
of the product x ^ is small (see Definition 13 . 51 and Lemma [3.111 below) . This 
is perhaps the most demanding part of our proof. Roughly speaking, we show 
that there is a distinguished subset of states reachable from each state in a fixed 
number of steps with probability bounded from 0. By applying Theorem 13. 6|, 
we obtain a complete invariant distribution on the product, i.e., in principle, 
we obtain a constant frequency of any non-trivial subset of states. From this 
we derive our results in a straightforward way. If GmxA is periodic, we use 
standard techniques for removing periodicity and then basically follow the same 
stream of arguments as in the aperiodic case. 

3.1 General state space Markov chains 

We start by recalling the definition of "ordinary" discrete-time Markov chains 
with discrete state space (DTMC) . A DTMC is given by a finite or countably in- 
finite state space S, an initial probability distribution over 5, and a one-step 
transition matrix P which defines the probability P(s, s') of every transion 
(s,s') e 5 X S" so that X^s'gS -^('^' '^') ~ ^ every s ^ S. In the setting of 
uncountable state spaces, transition probabilities cannot be specified by a tran- 
sition matrix. Instead, one defines the probabilities of moving from a given state 
s to a given measurable subset X of states. Hence, the concept of transition 
matrix is replaced with a more general notion of transition kernel defined below. 
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Definition 3.3. A transition Icernel over a measurable space (F, Q) is a function 
P -.T X g ^ [0,1] such that 

1. P{z, •) is a probability measure over (F, Q) for each z G F; 

2. P{-,A) is a measurable function for each A^Q (i.e., for every c G M, the 
set of all z eT satisfying P{z, A) > c belongs to Q ). 

A transition kernel is the core of the following definition. 

Definition 3.4. A general state space Markov chain (GSSMC) with a state space 
{T,Q), a transition kernel P and an initial probability measure fi is a stochastic 
process $ = $i, $2; ■ • ■ such that each $i is a random variable over a probability 
space {flq,, Tq,,V,s>) where 

• J7$ is a set of runs, i.e., infinite words over F. 

• J-<i, is the product a -field {^^q Q . 

• P^ is the unique probability measure over (fl^,J-^) such that for every 
finite sequence Aq, ■ ■ ■ , An € J-$ we have that V,s>{^o&Ao, ■ ■ ■ , $„G^„) is 
equal to 




fi{dyo) ■ P{yo, dyi) ■ ■ ■ P{yn^i, An). (2) 



• Each $i is the projection of elements of onto the i-th component. 

A path is a finite sequence zi ■ ■ ■ z„ of states from F. From Equation ^ 
we get that $ also satisfies the following properties which will be used to show 
several results about the chain $ by working with the transition kernel only. 

1. Ki.($o e Ao) -/i(Ao), 

2. P$($„+i G A I $„,..., $o) = P$($„+i G A I *„) = P($„,A) almost 
surely, 

3. P^{<^n+r,i e A I $„) = P™($„, A) almost surely, 
where the m-step transition kernel P™ is defined as follows: 

P^{z,A) = P{z,A) 
P'+\z,A)^ j^P{z,dy)-P\y,A). 

Notice that the transition kernel and the m-step transition kernel are analo- 
gous counterparts to the transition matrix and the fc-step transition matrix of a 
DTMC. 

As we mentioned above, our proof of Theorem 13.11 eniplovs several results of 
GSSMC theory. In particular, we make use of the notion of smallness of the 
state space defined as follows. 
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Definition 3.5. Let m e N, e > 0, and v be a probability measure on Q . A set 
C G G is (to, e, z^) -small if for all x G C and B £ Q we have that P"^{x,B) > 



GSSMCs where the whole state space is small have many nice properties, 
and the relevant ones are summarized in the following theorem. 

Theorem 3.6. IfT is {m, e, i')-small, then 

1. [Existence of invariant measure] There exists a unique probability mea- 
sure TT such that for all A G Q we have that 



2. [Strong law of large numbers] If h : T -^M. satisfies Jp h{x)n{dx) < oo, 
then almost surely 



Proof. The theorem is a consequence of stadard results for GSSMCs. Since F is 
(to, e, i/)-small, we have 

(i) $ is by definition (/^-irreducible for (p = v, and thus also ^-irreducible by [18| 
Proposition 4.2.2]; 

(ii) r is by definition also (a, e, z^)-petite (see [TBI Section 5.5.2]), where a is the 
Dirac distribution on Nq with a(jn) ~ 1, a{n) = for n ^ m; 

(iii) the first return time to F is trivially 1. 

ad 1. By (iii), F is not uniformly transient, hence by (i), (ii) and [18l Theorem 
8.0.2], $ is recurrent. Thus by [ISj Theorem 10.0.1], there exists a unique 
invariant probability measure tt. 

ad 2. By (i)-(iii) and [18l Theorem 10.4.10 (ii)], $ is positive Harris. Therefore, 
we may apply [HI Theorem 17.0.1 (i)] and obtain the desired result. 

ad 3. This follows immediately from iST, Theorem 8]. □ 





3. [Uniform ergodicity] For all x GT , A E Q, and all n £ N, 



sup |P"(x,A) -7r(A)| < (l-e)L"/™J 
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3.2 The product process 

The product process of Ai and A, denoted by x is a GSSMC with the state 
space r^x^ = SxQx (K>o)", where n = \X\ is the number of clocks of A. The 
cr-field over TmxA is the product cr-field GmxA = 2^^ (g) 2'^ (g) *B" where *B" is 
the Borel tr-field over the set (IR>o)"'- For each A G GmxAi the initial probability 
I^Mxa{A) is equal to X](s o)^a c*^o(s) (recall that ao is the initial distribution 
ofA^). 

The behavior of Al x ^ is depicted in Figure [H Each step of the product 
process corresponds to one step of Al and two steps of A. The step of the 
product starts by simulating the discrete step of A that reads the current state 
of M. and possibly resets some clocks, followed by simulating simultaneously the 
step of M that takes time t and the corresponding step of A which reads the 
time stamp t. 

Now we define the transition kernel PmxA of the product process. Let z = 
{s,q,h') be a state of T^^Ay and let (g, i^) be the configuration of A entered 
from the configuration (g, v) after reading s (note that P is not necessarily the 
same as ly because A may reset some clocks). It sufhces to define PMxAi^, ■) 
only for generators of GmxA and then apply the extension theorem (see, e.g., 
[5]) to obtain a unique probability measure Pmxa{z, ■) over (F^vix^Ai 0A4x.A)• 
Generators of GmxA are sets of the form {s'} x {q'} x I where s' ^ S, q' & Q 
and I is the product /i x • • • x /„ of intervals li in ]R>o. If q' q, then we define 
Pmxa{z, {s'} X {q'} X I) — 0. Otherwise, we define 

/•oo 

Pmxa{z, {s'} X {q'} X I) = P{s){s') ■ / fit) ■ li{D + t)dt 

Jo 

Here / = D(s, s') and li is the indicator function of the set I. 

Since PjvixAiz, ■) is by definition a probability measure over (TmxA, Gmxa), 
it remains to check the second condition of Definition 13.31 

Lemma 3.7. Let A G GmxA- Then Pmxa{'t A) is a measurable function, i.e., 
MxAis a GSSMC. 

A proof of this lemma can be found in Appendix IB. 11 Recall that by Def- 
inition [2131 VmxA is the unique probability measure on the product cr-field 
J^MxA — ^iLo^MxA induced by PmxA and the initial probability measure 

fJ-MxA- 



3.2.1 The correspondence between A^ x ^ and Ai 

In this subsection we show that A^ x ^ correctly reflects the behaviour of 
M. First, we define the d-^ measure for A^ x ^. (As the DTA A is fixed, 
we omit them and write d and instead of and d^, respectively.) Let 
a — {so,qQ,vo) (51,(71,1/1) • • • be a run oi A4 x A and q € Q a. location. For 
every i G No, let 1* (ct) be either 1 or depending on whether \i qi = q or not, 
respectively. We put 

A t \ V Er=ii«('^) 

dg(cr) — limsup 
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Lemma 3.8. There is a measurable one-to-one mapping ^ from the set of runs 
of AA to the set of runs of AA x A such that 

• ^ preserves measure, i.e., for every measurable set X of runs of M. we have 
that S,{X) is also measurable and Vm{X) = VMxAiCi^))) 

• ^ preserves d, i.e., for every run a of M and every q G Q we have that 
<iq{<y) is well-defined iff dq(^{a)) is well-defined, and dq{(7) —dq((^{a)). 

A formal proof of Lemma 13.81 is given in Appendix IB. 21 
3.2.2 The region graph of x ^ 

Although the state-space TmxA is uncomrtable, we can define the standard 
region relation ~ over TmxA with finite index, and then work with finitely 
many regions. For a given a G M, we use frac{a) to denote the fractional part 
of a, and int{a) to denote the integral part of a. For a, 6 £ R, we say that a and 
b agree on integral part if int{a) = int(b) and neither or both a, b are integers. 

We denote by -Bmax the maximal constant that appears in the guards of A 
and say that a clock x G X is relevant for v if v{x) < -Bmax- Finally, we put 

(si,gi,Z^l) - (S2,g2,i^2) if 

• si = S2 and qi ^ q2; 

• for all relevant x Cz X we have that vi{x) and i'2ix) agree on integral parts; 

• for all relevant x,y G A" we have that frac{h'i{x)) < frac{i'i{y)) iff 
frac{v2{x)) < frac{v2{y))- 

Note that is an equivalence with finite index. The equivalence classes of ~ 
are called regions. Observe that states in the same region have the same behav- 
ior with respect to qualitative reachability. This is formalized in the following 
lemma. 

Lemma 3.9. Let R and T be regions and z,z' G R. Then Pmxa{z,T) > iff 

PMy.A{z',T)>0. 

A proof of Lemma l3.9l can be found in [lOj . Further, we define a finite region 
graph GmxA = {V, E) where the set of vertices V is the set of regions and for 
every pair of regions R,R' there is an edge {R,R') G iff PmxaIz, R') > 
for some z E R (due to Lemma [3.91 the concrete choice of z is irrelevant). For 
technical reasons, we assume that V contains only regions reachable with positive 
probability m. M. x A. 

3.3 Finishing the proof of Theorem 13.11 

Our proof is divided into three parts. In the first part we consider a general 
region graph which is not necessarily strongly connected, and show that we can 
actually concentrate just on its BSCCs. In the second part we study a given 
BSCC under the aperiodicity assumption. Finally, in the last part we consider a 
general BSCC which may be periodic. (The second part is included mainly for 
the sake of readability.) 
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Non-strongly connected region graph 

Let Ci, . . . ,Cfe be the BSCCs of the region graph. The set TZi consists of all 
runs a oi M such that ^(w) visits (a configuration in a region of) Ci, where ^ 
is the mapping of Lemma 13.81 By applying the arguments of [TJ [10] , it follows 
that almost runs in x ^ visit a configuration of a BSCC. By Lemma [3.81 ^ 
preserves d and the probability VMiT^i) is equal to the probability of visiting Ci 
in X ^. Further, since the value of d does not depend on a finite prefix of a 
run, we may safely assume that x ^ is initialized in Ci in such a way that the 
initial distribution corresponds to the conditional distribution of the first visit 
to d conditioned on visiting d. 

In a BSCC Ci, there may be some growing clocks that are never reset. Since 
the values of growing clocks are just constantly increasing, the product process 
never returns to a state it has visited before. Therefore, there is no invariant 
distribution. Observe that all runs initiated in Ci eventually reach a configuration 
where the values of all growing clocks are larger than the maximal constant i3max 
employed in the guards of A. This means that Ci actually consists only of regions 
where all growing clocks are irrelevant (see Section 13.2.21) , because Ci would not 
be strongly connected otherwise. Hence, we can safely remove every growing 
clock X from Ci, replacing all guards of the form x > c or x > c with true and 
all guards of the form x < c ot x < c with false. So, from now on we assume 
that there are no growing clocks in Ci. 

Strongly connected &: aperiodic region graph 

In this part we consider a given BSCC Ci of the region graph GmxA- This is 
equivalent to assuming that GmxA is strongly connected and TmxA is equal 
to the union of all regions of GmxA (recall that GmxA consists just of regions 
reachable with positive probability in x ^). We also assume that there are 
no growing clocks (see the previous part). Further, in this subsection we assume 
that GmxA is aperiodic in the following sense. 

Definition 3.10. A period p of the region graph GmxA is the greatest common 
divisor of lengths of all cycles in GmxA- The region graph GmxA is aperiodic 
ifp = 1. 

The key to proving Theorem 13.11 in the current restricted setting is to show 
that the state space of Al x is small (recall Definition 13. 5|) and then apply 
Theorem 13.61 llj and ([2|) to obtain the required characterization of the long-run 
behavior of A^ x ^. 

Proposition 3.11. Assume that GmxA is strongly connected and aperiodic. 
Then there exist a region R, a measurable subset S C n € N, b > 0, and 
a probability measure k such that k{S) — 1 and for all measurable T <Z S and 
z g TmxA we have that RmxA^^tT) > b ■ k{T). In other words, the set F^vix^ 
of all states of the GSSMC M 'x A is (n, 6, k) -small. 

Sketch. We show that there exist z* £ yMxA^ « G N, and 7 > such that for 
an arbitrary starting state z G F^x^ there is a path from z to z* of length 
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exactly n that is "/-wide in the sense that the waiting time of any transition in 
the path can be changed by ±7 without ending up in a different region in the 
end. The target set S then corresponds to a "neighbourhood" of z* within the 
region of z*. Any small enough sub- neighbourhood of z* is visited by a set of 
runs that follow the 7-wide path closely enough. The probability of this set of 
runs then depends linearly on the size of the sub-neighbourhood when measured 
by K, where k is essentially the Lebesgue measure restricted to S. 

So, it remains to find suitable z*, n, and 7. For a given starting state 
z e r_A4x^, we construct a path of fixed length n (independent of z) that al- 
ways ends in the same state z* . Further, the path is 7-wide for some 7 > 
independent of z. Technically, the path is obtained by concatenating five sub- 
paths each of which has a fixed length independent of z. These sub-paths are 
described in greater detail below. 

In the first sub-path, we move to a S-separated state for some fixed 6 > 
independent of z. A state is i5-separated if the fractional parts of all relevant 
clocks are approximately equally distributed on the [0, 1] line segment (each two 
of them have distance at least 6) . We can easily build the first sub-path so that 
it is 5- wide. 

For the second sub-path, we first fix some region Ri. Since GmxA is strongly 
connected and aperiodic, there is a fixed n' such that Ri is reachable from an 
arbitrary state of F^nx^t in exactly n' transitions. The second sub-path is chosen 
as a (5/n')-wide path of length n' that leads to a (5/n')-separated state of i?i (we 
show that such a sub-path is guaranteed to exist; intuitively, the reason why the 
separation and wideness may decrease proportionally to n' is that the fractional 
parts of relevant clock may be forced to move closer and closer to each other by 
the resets performed along the sub- path). 

In the third sub-path, we squeeze the fractional parts of all relevant clocks 
close to 0. We go through a fixed region path Ri ■ ■ ■ Rk (independent of 2;) so 
that in each step we shift the time by an integral value minus a small constant c 
(note that the fractional parts of clocks reset during this path have fixed relative 
distances). Thus, we reach a state that is "almost fixed" in the sense that 
the values of all relevant clocks in z'^. are the same for every starting state z. 
Note that the third sub-path is c-wide. At this point, we should note that if 
we defined the product process somewhat differently by identifying all states 
differing only in the values of irrelevant clocks (which does not lead to any 
technical complications), we would be done, i.e., we could put z* — z'^.. We have 
neglected this possibility mainly for presentation reasons. So, we need two more 
sub-paths to fix the values of irrelevant clocks. 

In the fourth sub-path, we act similarly as in the first sub-path and prepare 
ourselves for the final sub-path. We reach a ^-separated state that is almost 
equal to a fixed state zi G Ri. Again, we do it by a J-wide path of a fixed length. 

In the fifth sub-path, we follow a fixed region path Ri ■ ■ ■ Re+m such that 
each clock not relevant in Ri is reset along this path, and hence we reach a fixed 
state z* G Re+rn- Here we use our assumption that every clock can be reset to 
zero (i.e., there are no growing clocks). □ 

Now we may finish the proof of Theorem 13. II By Theorem l3.6l pi). there is a 
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unique invariant distribution tt on Tj^^A- For every q € Q, we denote by Ag the 
set of all states oi M x A oi the form {s,q,v) £ Tj^xA- By Theorem 13.61 
for almost all runs cr of x ^ we have that d(i7) is well-defined and dq{a) = 
J^'^i^q)- By Lemma [3.81 we obtain the same for almost all runs of A^. 

Strongly connected &: periodic region graph 

Now we consider a general BSCC Ci of the region graph GmxA- Technically, we 
adopt the same setup as the previous part but remove the aperiodicity condition. 
That is, we assume that GmxA is strongly connected, TmxA is equal to the 
union of all regions of GmxAi a-nd there are no growing clocks. 

Let p be the period of GmxA- Li this case, A4 x Ais not necessarily small in 
the sense of Definition [23] By employing standard methods for periodic Markov 
chains, we decompose A4 x A into p stochastic processes ^o, . . . ,^p^i where 
each $fe makes steps corresponding to p steps of the original process A4 x A 
(except for the first step which corresponds just to k steps of x ^). Each 
is aperiodic and hence small (this follows by slightly generalizing the arguments 
of the previous part; see Proposition I3.13|) . Thus, we can apply Theorem 13.61 
to each $fc separately and express the frequency of visits to q in $fe in terms of 
a unique invariant distribution nk for Finally, we obtain the frequency of 
visits to (J in A4 X ^ as an average of the corresponding frequencies in 

Let us start by decomposing the set of nodes V of GmxA into p classes that 
constitute a cyclic structure (see e.g. [TTJ Theorem 4.1]). 

Lemma 3.12. There are disjoint sets Vq, . . . , Vp-i C V such that V — IJ^^g Vk 
and for all u,v £ V we have that {u,v) E E iff there is k £ {0,...,p — 1} 
satisfying u £ Vk and v £ Vj where j ~ {k + 1) mod p. 

For each k £ {0, . . . ,p — 1} we construct a GSSMC with state space 
^MxA = U_ReVfc ^ transition kernel PP(-, •) restricted to F^^^: a-nd an initial 
probability measure defined by /Ltfe(A) — J^^rMxA ^(^'^) ' ^''^(^, For each 
k, we define the discrete frequency d^ of visits q in the process Then we 
show that if d*^ is well-defined in we can express the frequency dg in x A. 

Note that for every run zq zi ■ ■ ■ of A^ x the word Zk Zp^k Z2p+k is a run 
of <i>fc. For a run a — (sqi QO: ^'o) (si, i^i) • • • , fc G {0, . . . ,p — 1}, and a location 
q £ Q, let define Ig^ii) to be either 1 or depending on whether qtp+k = q or 
not, respectively. Further, we put 



Assuming that each d*^ is well-defined, for almost all runs cr of A^ x ^ we have 
the following: 



dg (a) — lim sup 



1 i,k 
i=l '-q 



n 
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So, it suffices to concentrate on d^. The following proposition is a generalization 
of Proposition 13 . 1 II to periodic processes. 

Proposition 3.13. Assume that GmxA is strongly connected and has a periodp. 
For every k G {0, . . . — 1} there exist a region Rk EVk, a measurable Sk C Rk, 
Uk S 5fe > 0, and a probability measure Kk such that Kk{Sk) — 1 and for every 
measurable T C Sk and z € ^mxA have P2^'^^{z,T) > bk ■ Kk{T). In other 
words, $fe is {nk,bk, Kk)-small. 

By Theorem 13.61 (fT|). for every fc€{0,...,p — 1}, there is a unique invariant 
distribution iTk on F^x^ for the process ^k- By Theorem 13.61 ©J each d'^ is 
well-defined and for almost all runs a we have that d^(tT) — iTkiAq). Thus, we 
obtain 

^ fc=0 

4 Approximating DTA Measures 

In this section we show how to approximate the DTA measures for SMPs using 
the m-step transition kernel PjC^^xA oi Ai x A. The procedure for computing 
^MxA ^ sufhcient precision is taken as a "black box" part of the algorithm, 
we concentrate just on developing generic bounds on m that are sufhcient to 
achieve the required precision. 

For simplicity, we assume that the initial distribution ao of A4 assigns 1 to 
some Sq € S (all of the results presented in this section can easily be generalized 
to an arbitrary initial distribution). The initial state in x ^ is zg = (sg, qo,0). 

As we already noted in the previous section, the constant k of Theorem 13.11 
is the number of BSCCs of GmxA- For the rest of this section, we fix some 
^ l£ j l£ k, and write just C, TZ and D instead of Cj, TZj and Dj, respectively. We 
slightly abuse our notation by using C to denote also the set of configurations 
that belong to some region of C (particularly in expressions such as PjuxAiz, C)). 

The probability Vm {T^) is equal to the probability of visiting C in Ai x A. 
Observe that 

VM{n) = lim PX^^^{zo,C) 

l—^OO 

Let us analyze the speed of this approximation. First, we need to introduce 
several parameters. Let Pmin be the smallest transition probability in A^, and 
D{M) the set of delay densities used in M, i.e., D{M) = {D(s, s') | s, s' e S}. 
Let be the number of vertices (regions) of GmxA- Due to our assumptions 
imposed on delay densities, there is a fixed bound > such that, for all / G 
D{M) and a; G [0,i3inax], either f{x) > cj) or f{x) = 0. Further, f{x)dx 
is either larger than cj) or equal to 0. 

Theorem 4.1. For every i G N we have that 

PMm-PUxA{^o,c) < (^i ^ ^Einii^yy'^'^ 

where c = 4 • \V\. 
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Sketch. We denote by B the union of all regions that belong to BSCCs of Gmy-A- 
We show that for c = 4 • \ V\ there is a lower bound Pbound = (Pmin • cxi ■ ^/cY 
on the probability of reaching B in at most c steps from any state z € TmxA- 
Note that then the probability of not hitting B after i = m ■ c steps is at most 
(1 — Pbound)™'- However, this means that Pj^^j^{z,C) cannot differ from the 
probability of reaching C (and thus also from Vm (7?-)) by more than (l—pbound)"^ 
because C C B and the probability of reaching C from B \ C is 0. 

The bound pbound is provided by arguments similar to the proof of Propo- 
sition 13.111 From any state z we build a (5-wide path to a state in B that has 
length bounded by 4 • \V\ such that 5 = Pmin ■ cj) • 1/c. The paths that follow 
this 5-wide path closely enough (hence, reach B) have probability pbound- D 

Now let us concentrate on approximating the tuple D. This can be done by 
considering just the BSCC C. Similarly as in Section |3l from now on we assume 
that C is the set of nodes of GmxA (i-S-, GmxA is strongly-connected) and that 
^MxA is equal to the union of all regions of C. 

As in Section [3l we start with the aperiodic case. Then, Theorem 13.61 ([3l) 
implies that each Dq can be approximated using Pj^^j^{u, Aq) where u is an 
arbitrary state of Tj^xA and Aq is the set of all states of x ^ of the form 
(s,g, z^). More precisely, we obtain the following: 

Theorem 4.2. Assume that GmxA is strongly connected and aperiodic. Then 
for all i ^N, u ^ ^MxA, q & Q 

\Dq- PMxAiu,Aq)\ < ) ) 

where r = 

Proof. From the proof of Proposition l3T3l (for details see Appendix [C|) . we obtain 
that TmxA is (m, £, K)-small with m < r and e — ( ^""""^^ )'', and the result 
follows from Theorem [3761 (l3l'). □ 

Now let us consider the general (periodic) case. We adopt the same notation 
as in Section m i.e., the period of GmxA is denoted by p, the decomposition of 
the set y by Vb, . . . , Vp-i (see Lemma [3.12p . and r^^_4 denotes the set Uflev^ ^ 
for every k & {0, . . . ,p — 1}. 

Theorem 4.3. For every i G N we have that 



where Uk £ F^^^^ and r ^ 1^1 J . 

Proof. Due to the results of Section [3] we have that Dq — i • X]fe=o ""fcl^?)' 
where tt^ is the invariant measure for the fc-th aperiodic decomposition $fc of 
the product process A4 x A (i.e. tt^ is a measure over F^^^). From the proof 
of Proposition 13.131 (for details see Appendix [C|) . F^^^ is (m, e, K)-small with 
m <r and e ~ ( ^""^^° )*", and the result follows from Theorem 13.61 ([51) applied 
to each F^^_^ separately. □ 
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5 Conclusions 



We have shown that DTA measures over semi-Markov processes are well-defined 
for almost all runs and assume only finitely many values with positive probabil- 
ity. We also indicated how to approximate DTA measures and the associated 
probabilities up to an arbitrarily small given precision. 

Our approximation algorithm is quite naive and there is a lot of space for 
further improvement. An interesting open question is whether one can design 
more efficient algorithms with low complexity in the size of SMP (the size of 
DTA specifications should stay relatively small in most applications, and hence 
the (inevitable) exponential blowup in the size of DTA is actually not so prob- 
lematic). 

Another interesting question is whether the results presented in this paper 
can be extended to more general stochastic models such as generalized semi- 
Markov processes. 
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A Proof of Corollary [33] 



COROLLARY l3.2l is well-defined for almost all runs of M.. Further, there are 
pairwise disjoint sets TZi, . . . , TZk of runs in M such that V{TZi U • • • UTZk) — 1, 
and for every 1 < j < K there is a tuple Cj such that c-^(cr) — Cj for almost all 

Most of the proof has aheady been presented in Section |3l It remains to 
prove that for ahiiost all runs a of TZj we have 



J2peQ Y^ses ■ ^j,(s,p) 



To simplify om" notation we write I?s,p and 1* ^ instead of -Dj p) and 
respectively, li Dq ^ then clearly both sides of Equation ([3]) are 0. Assume 



that Dq>0. 



We prove that for almost all runs a of TZ 



■J' 



' n— >cxD fi 

J2Y.^^-D^.p- (5) 

— ^ ^ — ^ n— ^oo n 

p<£Q seS 

which proves Equation ([3]) because 

By the strong law of large numbers, for almost all runs ct of 7?. we have 

Es = lim ,^ , r (6) 

for all s G S* and p £ Q satisfying D^^p > (note that waiting times in s do not 
depend on p). Let a be a run of TZ which satisfies Equation ([6|) for all s e 5 and 
p £ Q where D^.p > and such that d-^^'^ is well-defined for a. 
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For every p ^ Q we have that X^ses ■ ^s,p is equal to 



Ehm 



ELir-(a)-n,,(a) 



Elim 
r), — i-rx 



' n— >-oo 
sG5 



which proves Equation (|4]). Also J2peQ J2ses ■ Ds.p is equal to 

^— ^ n— >cxD 77, n— >oo n 

n—^oo Ji 

which proves Equation ^ and finishes the proof. □ 



B Proofs of Section [3721 
B.l Proof of Lemma 13.71 

Lemma 13.71 Let A e Qm^A- Then Pm^a{'tA) is a measurable function, i.e., 
MxAis a GSSMC. 

Proof. To prove this lemma, it is sufficient to show that ^lMxyt(',^) is a mea- 
surable function from F^vix^ to [0, 1] where A ranges (only) over the generators 
of QmxA, i-e. A = {s'} X {q'} x I where s' G S, q' € Q, and I = Yix^x ^uch 
that Ix is an interval for each x G X (see, e.g., |15l Lemma 1.37]). 

As the sets S and Q are finite, our goal is to show that a function 
PMxA{{s,q, X W) X I) is measurable for s,s' G S, q,q' £ Q, and a 

product of intervals I. 

The rest of the proof is based on the fact that a real valued function is 
measurable, if it is piecewise continuous. Hence, we finish the proof showing 
that the function PMxAiis,q,-),{s'} x {q'} x I) is piecewise continuous when 
we fix valuation of all clocks but one. Formally, we fix a valuation v and a clock 
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X and show that the fohowing function of a parameter u £ M>o is piecewise 
continuous. 

/•oo 

PMy.A{{s, q, v[x := u]), {s'} x {g'} x I) = 5,,,- .P(s)(s') • / /(t) ■ li{v + t)dt 

Jo 

where 

• iy[x := u] is the valuation ly where the value of the clock x is set to u; 

• S is the Kronecker delta, i.e., Sqiq — 1 ii q' = and 0, otherwise; 

• f — D(s, s') is the delay density function for (s, s'); 

• {q^v) is the timed automaton successor of the state {s,q^v[x :— u]), i.e., 
A((s,q,iy[x := u])) = {q,D); 

• li is the indicator function of the set I, i.e., = 1 if i^' G I, and 0, 
otherwise. 

The function P(s)(s') is constant (recall that s and s' are fixed). Due to the 
standard region construction for A, it holds that q is piecewise constant and D 
is piecewise continuous with respect to u. 

Let u be in one of the finitely many intervals where Sqiq is constant and the 
valuation D changes continuously, i.e. the automaton A uses the same transition 
for all u of this interval. As li is the indicator function and I is a product of 
intervals, it holds that 

/■oo nh{u) 
5q.q ■ P(S)(S') • / fit) ■ h{D + t)dt = 5q,q ■ P(s)(,s') • / f (t) dt 

Jo Ja.{u) 

where a(u) and b(w) are continuous functions of u and so /J^^^"'' f{t) dt is also 

a continous function of u (recall that f{t) dt = 1). Therefore, the function 
PMxAiiSjq,v[x := u]),{s'} x {q'} x I) is a piecewise continuous function of u 
and Pmxa(:; A) is a measurable function, i.e., x ^ is a GSSMC. □ 

B.2 Proof of Lemma ISTSl 

Lemma 13.81 There is a measurable one-to-one mapping ^ from the set of runs 
of Ai to the set of runs of A4 x A such that 

• ^ preserves measure, i.e., for every measurable set X of runs of M we have 
that ^(X) is also measurable and Vm{X) =^ 'PMxAiS.i^)), 

• ^ preserves d, i.e., for every run a of M and every q ^ Q we have that 
dq{a) is well-defined iff dq(£^{a)) is well-defined, and (iq{a) — dq{$^{a)). 
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Proof. First, we define the function ^. We use auxiliary functions £,0 : S 
^MxA that maps the initial states and a function : Tj^xA x (5' x R>o x 5) -> 
^MxA that maps transitions. First, we set £,o{s) — (s,(7o,0) where go is the 
initial location and is the zero vector. Next, let s,s' G S, t d R>o and 
z — {s",q,v) £ FjVix^- We define (s, s')) — {s',q',v' + t) such that 

A{z) = {q'y). 

For a run a = so^osiii • • • , we use these two functions to set ^(cr) = ^0-^1-^2 • • • 
such that ^o(so) = zq and for each i € Nq it holds that £^{zi, {si,ti, Si+i)) = Zi+i. 
We need to show the following claims about the function £. 

Claim B.l. Let a he a run of M. We have for any q G Q that dq(a) is 
well- defined if and only if dq{£^{a)) is well-defined, and dq{a) = dq{£{a)). 

Let a — 50^051^152^2 • • • be a run of Ai. Let us fix a location q € Q. Recall 
the Figure [2j The run of A over tr is a sequence 

= iqQ,vo)sQiqi,vo)to{qi,vi)si{q2,vi)ti{q2, 1^2)32 ■■■ ■ 

The corresponding run of the product is 

^((7) = (so,'7o,i^o)(5i,gi,i^i)(52,g2,i^2)-- - ■ 

The values dg(cr) and dq{£{a)) are limit superior of partial sums of ratio of q 
in a sequence of locations. For dq{a) the sequence is Q^{a),Q^{a), Q^{a), . . . = 
Qi)l2,q3, ■ ■ ■ (recall that Q^{ct) is the location entered after reading the finite 
prefix 5o to • • • Si) and for dg(^(<T)) the sequence is also gi, 92, '?3, ■ • •■ Hence, we 
get that dq{a) is well-defined iff dq{£^{(j)) is well-defined and dg(cr) = dq{S^{(j)). 

Claim B.2. For any measurable set X of runs of M., the setS^{X) is measurable. 

Recall that by TZ{B) we denote a cylinder of runs that follow the given 
template B. Let X be a set of runs such that X = Tl{B) for some template 
B = sqIq ■ ■ ■ 5„/„, i.e. X is from the generator set. We can cover the image of X 
by cylinders composed of basic hybercubes. By decreasing the edge length of the 
hypercubes to the limit, we then get a set that equals the image of X. For fc £ N 
and V e (No)''^' we denote by a set of valuations J|^g^[v(x)/fc, (v(a;) + l)/fc]. 
The set of all cylinder templates composed of basic hypercubes of precision k is 

Uk - {Ao • • • A„ I A, = {5 J X {q,} X C^^ , 5, e 5, <z, e g, V, e (No)l^l } 

A run a — zqZi ■ ■ ■ of x ^ is in TZ{Aq . . . An) if for each < i < n we have 
Zi Cz Ai. It is easy to show that 

ax) - n U{^(^) \c^u,, n{C) n £{x) + 0} 

feeN 

hence, £{X) is a measurable set. By standard arguments we get the result for 
any measurable X. 

Claim B.3. For any measurable set X of runs of A4 x A, the set S^~^{X) is 
measurable. 
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The arguments are similar as in the previous claim. Let y be a set of runs 
such that Y = TZ{C) for some template 

C = {so} X {qo} X 4,0 • • • {Sn} X {qn} X 4,„, 

xex xex 

i.e. Y is from the generator set. By we denote an interval [i/k, {i + l)/fc]. 
The set of all cylinder templates in M composed of basic lines of precision k is 

Tk = {solt ■ ■ ■ Suit |siG5,i„GNo} 

Again, it is easy to show that 

r\Y) = fi \J{n{B) I B G Tfe,7^(B) n r'(^) 0} 

feeN 

hence, ^~^{Y) is a measurable set. Again, by standard arguments we get the 
result for any measurable X. 

Claim B.4. For any measurable set X of runs of A4, we have Vm{^) = 
We define a new measure V'j^ over runs of M. by 

V'M{X)=VM.A{m) 

for any measurable set of runs X. First wc need to show that Vj^ is a probability 
measure, i.e. 7^^(0) = 0, V'j^{TZm) ~ 1, and for any collection of pairwise 
disjoint sets Xi. . . . ,X„ we have ^>((U"=i ^i) = X]"=i "PMi-^i)- The first and 
the second statement follows directly from the definition of ^, the third statement 
follows from the fact that VmxA satisfies this property and that ^-image of 
disjoint sets are disjoint sets of runs which can be easily checked. 
Let B = sqIq ■ ■ ■ Snin be a cylinder template. We show 

v'M{n{B)) = VM{n{B)). 

We obtain — Vm by the extension theorem because Vj^ and Vm coincide on 
the generators. From the definition of "P^ we get the claim. From the definition 
of semi-Markov process, we have 

n ™ 

Vm{TI{B)) = ao(so) ■ nP(si)(«i+i) • / fi{U)dU 

where fi = D{si, Si^i) is the density of the transition from Si to .Si+i. Now 
we turn our attention to the product. For the fixed template we define a set 
■^0 = {(sO) 90, 0)} and a sequence of functions Ni, . . . , Nn such that 

Ni+i{zi) = {^^{zi,{si,ti,Si+i)) I ti G li}. 
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For an interval / — [a, b] and valuation v we define C(/, v) to be a hypercube of 
edge length b — a starting at v + a, i.e. C{I,i') — Ilxe a; + a, ^{x) + b]. Now 
for each z £ No the conditional density 

where N^+i{z) = Ug^+iegi^'+il ^ x C'(/,,i>) where A{z) = {q,D). The 

equality holds because iVi+i($i) C iV,+i($,) and VMxA{^i+i G (iVi+i($i) \ 
iVi_|_i($i)) I = 0. Indeed, the probability of hitting anything else but the 
diagonal of the hypercube Ni^i{<^i) is clearly 0. Because Ni^i{z) is for each z 
a union of basic cylinders for that we have explicit definition of the transition 
kernel, we have 



/•oo 

y^5gq-V{si){s,+i)- / h{t)-lc{u,u){i> + t)dt 



where S is the Kronecker delta, i.e., 6qq = 1 ii q = q, and 0, otherwise. Notice 
that here q and P are random variables such that A{^i) = {q, v). The rest of 
the formula after 5qq does not depend on g, we can write 

/•oo 

Jq 

Furthermore, hitting the hypercube C{Ii, v) equals to waiting for a time from li 
= Pis,){s,+,)- f,{t)dt. 

Hence, the conditioned probability is a constant random variable that does not 
depend on $i. Finally, 

V'M{n{B)) =VMxA{m{B))) 

= VMxA{^a^No,^^ eiVi($o),---,*n G^n(*n-l)) 
= VMxAi'i'n e 7V„($„_i) I $0, • ■ • , $n-l) ' ' • ' 

• Vmxa{^i e A^„($o) I $o) • Vmxa{^q e iVo) 



n fp(so(s,+i) • / f,{u)dt, 



Q!o(so) 



which concludes the proof. □ 

C Proofs of Section [3:3] 
C.l Proofs of Proposition [3.111 
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Proposition [STTTl Assume that QmxA strongly connected and aperiodic. 
Then there exists a region R and SCR^nGN,b>0, and a probability measure 
/i such that /i(S') — 1 and for every measurable T C S and z G F^x^ we have 
PJl^^y^{z,T) > b-^(T). In other words, the setTj^xA of all states of the GSSMC 
A4 X A is {n,b, small. 

Proof. Follows easily from Proposition 13. 13l bv considering the period p equal to 
1. □ 

C.2 Proofs of Proposition SHI 

Proposition 131 Assume that GmxA is strongly connected and has a period 
p. For every k € {0, . . . ,p — 1} there exists a set of states Sk, rik G N, bk > 0, 
and a probability measure Kk such that Kk{Sk) — 1 and for every measurable 
and z G F^^^ we have P^^_^{z,T) > bk ■ Kk(T). In other words, $fe 
is {nk,bk,Kk) -small. 

In the following text we formulate the definitions and lemmata needed to 
prove the proposition. The actual proofs of the lemmata are in next subsections 
(grouped by proof techniques) . 

Let us fix a A: G {0, ... ,p — 1}. We show that there is a state z* G ^mxa 
such that for each starting state z G ^m^a there is a path z ■ ■ ■ z* of length 
rifc • p that is 6 -wide. For a fixed (5 > 0, it means that the waiting time of any 
transition in the path can be changed by ±S without ending up in a different 
region in the end. Precise definition follows. 

Definition C.l. Let z = (s, q, v) and z' = (s', q' , v') be two states. For a waiting 
time t G ]R.>o we set z^z' if A{z) — {q' ,v) and v' = D -\-t. We set z^z' if 
z\ z' for some t G M>o and call it a feasible transition. 

For 6 > 0, we say that a feasible transition z^ z' is S-wide if for every x G X 
relevant for v' we have frac{vi(x)) G [5, 1 — 5]. 

Let z\ - ■ ■ Zn be a path. It is feasible if for each I < i < n we have that 
Zi^Zi^i. It is (5-wide if for each 1 < i < n we have that — > Zi+i is a 5-wide 
transition. 

By next lemma, we reduce the proof of Proposition 13.131 to finding (5-wide 
paths from any z to the fixed z*. 

First, we recall the following notation that is necessary for analyzing the 
computational complexity. Let Pmin denote the smallest probability in A4 . Fur- 
ther, let us denote by D{M) the set of delay densities used in M, i.e. T>{M) = 
{D(s, s') I s,s' G S}. From our assumptions imposed on delay densities we 
obtain the following uniform bound cj) > on delay densities of For 
every / G 2)(A^) and for all x G [0, Bmax], either f{x) > or f{x) = 0, and 
moreover, f{x)dx > c or equals 0. 

Lemma C.2. For every (5 > and n > 1 there is a probabilistic measure k and 
b > such that the following holds. For every 5-wide path a = zqZi ■ ■ ■ Zn, there 
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is a K-measurable set of states Z with k{Z) = 1 such that Zn & Z and for any 
measurable subset Y C Z it holds Pj!^^j^{zi,Y) > b ■ k{Y). 
Moreover, we can set b = (pmin • cj) • 5/n)'^/^/\X\. 

Now it remains to find a state z* and a i5-wide path to z* for any z. Such 
path is composed of five parts, each having a fixed length. The target state z* is 
then the first state where all these paths from all starting states z meet together. 

In the first part, we move to a 5' -separated state for some 5' > 0. 

Definition C.3. Let i5 > 0. We say that a set X C R>o is 5-separated if for 

every x,y € X either frac{x) ~ frac(y) or \frac{x) — frac(y)\ > S. 
Further, we say that (s, q, ly) G TmxA is 5-separated if the set 

{0} U {i'{x) \ X € X,x is relevant for v} 

is 5-separated. 

Now we can formulate the first part of the path precisely. 

Lemma C.4. There is 5 > and n € N such thai for any Zi € T_mxA there is 
a 5-wide path zi ■ ■ • Zn such that Zn is 5-separated. 
Moreover, we can set n = -Bmax • l-^l and 6 = 1/{2{\X\ + 2)). 

At the beginning of the second part, we are in a ^-separated state zi in some 
region R G Vk' for some k' G {0, . . . ,p — 1}. For the given k' , we fix a region 
Ri G Vk'. Due to strong connectedness, reaching i?i is possible from any state 
in Vk' in a fixed sufficiently large number of steps n'. By a path of length n' 
that is (5/n')-wide, we reach a ((5/n')-separated state in The separation and 
wideness decreases with length because the fractional values of relevant clock 
may be forced to get closer and closer to each other by resets on the path to 
The reason for the first part of the path was only to bound the wideness of the 
second part. 

Lemma C.5. Let the region graph GmxA &e strongly connected and letp be the 
period of GmxA- Let k € {0, . . . , p — 1} , 6 > and R G Vk be a region. Then 

there is n £ N such that for every 5-separated z\ G rvix^t there is a (5/n)-wide 
path zi - ■ ■ Zn such that Zfi IS {5 /n) -separated and Zn G R. 
Moreover, we can set n = [|F|^^"l^l~-^/6j - p. 

In the third part, we squeeze the fractional values of all relevant clocks close 
to 0. We go through a fixed region path Ri ■ ■ ■ Rk such that in each step we 
shift the time by an integral value minus a small constant c. This way the reset 
clocks are fractionally placed to and the other clocks decrease their fractional 
values only by the small constant c. Since we go through a fixed region path, 
we have a fixed sequence of sets of clocks Xi, . . . , Xk-i reset in respective steps. 
Hence, the fractional values of clocks reset during this path have fixed relative 
distances. For any starting state z[ we reach a state z'f. that almost equals a 
fixed "reference" state Zk G Rk- 

Definition C.6. Let z, z' G V j^^^. We say that state z almost equals state z' 
if z ^ z' and each clock relevant in z has the same value in z and z' . 
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Notice that clocks not relevant in Zf. may still have different values. 

Lemma C.7. Let R he a region. For each 5 > Q there is 5' > 0, n G N and 
z' G r_A4xyi such that for every S-separated zi G R there is a 6' -wide path zi • • • z„ 
such that Zn and z almost equal. 

Moreover, we can set n — iJmax + 1 and S' = 5/{Ba\iac + 2). 

In the fourth part, we somewhat repeat the first part and prepare for the fifth 
part. We reach a (5-separated state that is almost equal to a fixed state zi € Ri- 
Again, we do it by a (5-wide path. 

Lemma C.8. Let z be a state. There is a S > 0, n E Nq, and z' such that 
for any state Zi almost equal to z there is a 6 -wide path Z\ - ■ ■ z„ such that z„ is 
b -separated and z„ almost equals z' . 

Moreover, we can set n — B^i^^ ' l-^l and S = 1/(2(|A'| + 2)). 

In the fifth part, we go through a fixed region path Ri ■ ■ ■ Ri+m such that 
each clock not relevant in Ri is reset during this path and hence we reach a fixed 
z* € Ri+m- Such path exists from the assumption that it is possible to reset 
every clock. The (arbitrary) values of clocks not relevant in i?/ do not influence 
the behavior of the timed automaton before their reset and we indeed follow a 
fixed region path. Furthermore, we can stretch the path to arbitrary length so 
that the length of the whole path is a multiple of the period p. Again, the fifth 
part of the path is i5/n"-wide where n" is the number of steps. 

Lemma C.9. Let the region graph GmxA be strongly connected. Let S > 0. Let 
z be a 6 -separated state. Then there is n G N, such that for any n' > n there is 
a state z* such that the following holds. For any state zi almost equal to z there 
is a {d/n)-wide path Zi - ■ ■ Zn' such that Zn' ~ z* . 
Moreover, we can set n= \y\ ■ \X\. 

Now we can finally prove the main proposition. 

of Proposition[3jE We fix fc e {0, ... ,p - 1}. By Lemmata [C31 EH EZl 
IC.8[ and IC. 91 we get for any state zi e ^mxA ^ ^-wide path zi ■ ■ ■ Zx of length 
X — Hk ■ p such that 

X = Bn,ax • \X\ + M + (Bn,ax + 1) + S„,ax ■ \X\ + \V\ ■ \X\ + C < 2 ■ M 
6 = ^ . , > 



((S„,ax + 2) • 2{\X\ + 2) • A/ - 4 • B^ax • \X\ ■ M 

where M = I^I~^)/6J -p and c < p is the constant such that x is a multiple 

of p (we stretch the path by c in the fifth part). Therefore, by Lemma [0.21 $fc 
is (nfc,&fc,Kfc)-smaU for nk < [\V\^^''\^\-^ \ ■p< =; r and 



\ 1 
Pmin ■ C2) \ J- 



bk = (Pmin • CJ) • hjxY l^/\X 
> 

> 



V M3 / - V r / 
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Notice that in the calculation above we ignore the cases of trivially small region 
graphs with less than 3 vertices. □ 



From the proof we directly get the following bound on constants. 

Corollary C.IO. Assume that GmxA is strongly connected and has a period p. 
For every k £ {0, . . . ,p— 1} we have is (n, b, n)- small for some n < r divisible 
by p, and b = (pmin • /rY , where r — '^'j . 

C.2.1 Proof of Lemma [C^ 

Lemma IC.2I For every 5 > Q and n > 1 there is a probabilistic measure k and 
6 > such that the following holds. For every 5 -wide path a = zqZi ■ ■ ■ Zn, there 
is a K-measurable set of states Z with n{Z) ~ 1 such that z„ e Z and for any 
measurable subset Y CI Z it holds PjUxA^-^'^^ Y) ^ b ■ k{Y). 
Moreover, we can set b = (pmin • • S/n)"/ 

Proof. Recall that we assume that all delays' densities are bounded by some 
C2) > in the following sense. For every d G S and for all x € [0, B], d{x) > 
or equals 0. Similarly, d{x)dx > c-s or equals 0. 

Let cr = zozi ■■■Zn = (so,9o,t'o)(si,9i,'^i) • • • isn,qn,Vn)- For 1 < I < 71, let 
ti be the waiting times such that Zi-i Zi, and let Xi = {x G X \ A{zi^i) = 
{q, v), ^{x) = 0} be the set of clocks reset right before waiting ti. 

For £ > 0, we define an e-neighbourhood of a to be the set of paths of 
the form zq ^ {si,qi,v[) ■ ■ ■ ^ (sn, t/lJ where t- £ {ti — e,t + e). Due to S- 
separation of cr, all paths of its (S/n- neighbour hood are feasible. Considering 
this (5/n- neighbour hood, the set of all possible v'^s forms the sought set Z. We 
may compute this set as follows. We define a mapping : (— e,e)" — IR>o 
so that aaiCij ■ ■ ■ Xn) = i^'n for — ti + Q. This can be done by setting 
aa-iCi, ■ ■ ■ , Cn){x) = Ylr^i^i + CO J whcrc the clock x was reset in the r^th step 
for the last time in <t, i.e. r^ — max{i | x € Xi}. Obviously, is a restriction 
of a linear mapping. Therefore, aa{{—s, e)") is an open rhombic hypercube of a 
dimension 1 < d < |'-f |. Due to the last summand, it has a positive K^j-measure. 
(Here is the standard Lebesgue measure on the d-dimensional affine space that 
contains Q;cr((— e, e)")- Equivalently, it is the d-dimensional Hausdorff measure 
multiplied by the volume of unit d-ball.) 

We set Z := a^{{-5/2n,S/2n)"). Thus, for every z e Z there is a 5/2- 
separated path r from zq to z. We need to construct b > such that for 
all Y C Z,we have P]i^^j^{zo,Y) > bKd{Y)/Kd{Z) =: bK{Y). It is sufficient to 
prove this for generators of the same topology. We pick the generators as follows. 
For z e Z and e < S/2n, we denote Y{z,e) = acr{{~S/n,5/n)"-) n Cz,6 , where 
Cz,6 is a hypercube with dimension \X\ and size e centered in z. Clearly, the 
set of all Y{z,e) C Z form a generator set. We now construct 6 > so that for 
every such Y Y{z,e) we have PJl^^J^{zo,Y) > budiY) / Hid{Z) . To this end, 
we prove later on that 

PMxAi-^^,Y) > (p„,incs/n)"<5""'^e^ (7) 
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Since Kd{Y) < ^/\X\ ■ (e)'' and Kd.{Z) > {S/nY, we can set b = 

(PminCZ)(5/n)"/y|A^. 

It remains to prove ([T]). Let ki,...,kd be the elements of {r^ \ x e X} 
in the increasing order, and £i, . . . ,£n-d the remaining nmxibers in {1, . . . , n}. 
Note that since a is hnear, a^^{Y) is A„-measurable (A„ denotes the standard 
Lebesgue measure on R"). Intuitively, if we want to make clock x hit Y, it is 
sufficient to adjust the waiting time after the last reset of x. Let Y\Xi denote 
the projection of Y to coordinates in Xi (setting other components to zero). 
The first equation makes use of the facts that (1) all components of each point 
of Y\Xi have the same value (because F is a subset of image of aa) and (2) 
when factoring out all (identical) components but one of each Xi, the image of 
y is a d-hypercube (due to the intersection with ClJ), so we can use projections 
independently. 



Pmxa(.ZO,Y) > / {PminCTlTdXn 
{PminCXiY 



a^(0,...,0,dfc^,...,d„)[Xfc^ey|Xfc^ J a^{0,...flAu^,...,d„)\Xk^eY\Xk^ 
(5/2n pS/2n pe/2n pe/2n 



&/2n J -S/2n J -e/2n J-e/2n 



n — d d 

dCki ■ ■ ■ dCkadCi^-d ■ --dCei 
= {PminCvr{S/nr-''{e/nf = {pnunCv /nTS^-^e'' □ 

□ 

C.2.2 Proofs of Lemmata [Clil and [CTsl 

Lemma IC.4[ There is 6 > and n G N such that for any z\ £ ^ mxA there is 
a 5-wide path zi • • • z„ such that 2„ is S-separated. 
Moreover, we can set n — Bmax • (|<-f | + 2) and S — 1/(2(|A'| + 2)). 



Proof. To simplify the argumentation we introduce a notion of a r-grid that 
marks r distinguished points (called lines) on the [0, 1] line segment. In the 
proof we show that we can place fractional values of all relevant clocks on such 
distinguished points. Let r G N. We say that a set of clocks y C X is on r-grid 
in z if for every x & y relevant in z we have frac{v{x)) = n/r for some < n < r. 
For < n < r, we say that the n-th line of the r-grid is free in z if there is no 
relevant clock in the l/2fc-neighborhood of the n-th line, i.e. for any relevant 
x & X we have frac{u{x)) ^ {n/r — l/2r, n/r + l/2r). 
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Let r = |X| + 2. We inductively build a l/2r-wide path zi ■ • • z„ where 
n — Bynax ■ r. The set is on r-grid in zi. We show that if a set is on r-grid in 
state Zi, there is a l/2fc-wide transition to Zi^i such that U Z) is on r-grid in 
Zi+i where Z is the set of clocks newly reset in Zi. There are |X| + 2 lines on the 
grid and only |X| clocks. At least two of these lines must be free. Let j he 
such a line. Let t be a waiting time and Zi+i a state such that frac{t) = 1 — j/r 
and Zi — >■ Zj+i. Such waiting time must be indeed possible because the interval 
where the density function of any transition is positive has integral bounds. The 
transition Zi^Zij^i is l/2r-wide because the line j is free in Zi. Furthermore, 
the set U Z) is on r-grid in Zi+i because the fractional value of each clock 
that was previously on r-grid was changed by a multiple of 1/r. The newly reset 
clocks have fractional value 1 — j/r which is again a multiple of 1/r. 

Next, we show that X is on r-grid in z„. Clocks reset in this path on r-grid 
in z„. The remaining clocks are all irrelevant because the path of Bmax • f steps 
takes at least Bmax time units. Indeed, each transition in this path takes at least 
1/r time unit. According to the definition, X is on r-grid in z„. Hence, the state 
Zn is 1/r-separated because the distance between two adjacent grid lines is 1/r. 
By setting 5 ~ l/2r we get the result. □ 

Lemma IC.8I Let z he a state. There is a 5 > 0, n ^ Nq, and z' such that 
for any state zi almost equal to z there is a 6 -wide path zi • • • z„ such that Zn is 
S -separated and z„ almost equals z' . 

Moreover, we can set n = i?max • l-^l o,nd 5 — 1/(2(|A'| -t- 2)). 

Proof. Let us fix a state zi almost equal to z. By Lemma IC. 41 we get a (5- wide 
path zi . . . z„ such that z„ is (S-separated. 

Notice that for a fixed state z, control state s and time t there is a unique 
location q and valuation v, hence a unique state z' ~ (s, g, v) such that z -t- z'. 

Let ii, . . . , tn-i be the waiting times and si, . . . , s„ the control states on the 
path zi, . . . , z„. For any zi almost equal to zi we can build using the same waiting 
times and control states a path zi • • • z„ . It is easy to see that for two almost 
equal states z,z a control state s and a time t > the states z',z' determined 
by s and t are also almost equal. Inductively, we get that z„ is almost equal to 
z„. It also holds that z„ is (5-separated since (5-separation is defined only with 
respect to relevant clocks. □ 

C.2.3 Proofs of Lemmata [CTsl and [CTqI 

For the proof of Lemma I C. 5 1 we need the following result from graph theory. 

Lemma C.ll. Let G he a strongly connected and aperiodic oriented graph with 
N > 2 vertices. Then for each n > \_N'^^'^^~^ /6\ , there is a path of length 
precisely n between any two vertices of G. 

Proof. It is a standard result from the theory of Markov chains, see e.g. [2^ 
Lemma 8.3.9], that in every ergodic Markov chain there is no such that between 
any two states there is a path of any length greater than riQ. In the following, 
we give a simple bound on no. 
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Let u, V be vertices. By aperiodicity, there are C cycles on u of lengths 
ci, . . . , cc < N with gcd(ci, . . . , cc) = 1- Thus by Bezout's identity, there are 
rrii G No such that 1 = J2i=i "^iQ. Hence also 1 = J2i=ii''^i + h/ci ■ Y[j=i 
for any ki + ■ ■ ■ + kc = 0. Therefore, 1 = X]i=i "i'^i with some > > 
-1/c, ■ nf=i Cj for i < C and nc > 0. By [TTl Theorem A. 1.1], 

no can be chosen 

TV + P{P — 1), where P = J2fSi \ni\ci, i.e. the absolute value of the negative 
part of the sum. Note that P < (C - 1)N'^ . 

Let ci have F different prime factors. Then C2 can be chosen indivisible by 
some of the factors. Then C3 can be chosen indivisible by some of the remaining 
factors and so on. Therefore, we can choose a so that C < F + 1. By [19l 
V.lS.l.b], for the number uj{N) of distinct prime factors of N, we have F < 
uj{N) < 1.391niV/lnlniV. Hence P < 1.391n7V/lnlniV ■ iVi+i-39i"^/i"i"^ and 
thus no < TV* '"^-76. 

□ 

Both proofs of Lemmata IC.5I and IC.9I use a technique expressed by the next 
lemma. 

Lemma C.12. Let S > 0, n G N, zi be a 6-separated state and ziz'2---z'^ 
be a feasible path. Then there is a {6/n)-wide path Z1Z2 ■ • ■ Zn such that Zn is 
(S/n)- separated and for each 1 < i < n we have Zi z'^. 

Proof. For simplicity, we first transform this path into a (5/2"- wide one. We then 
show how to improve the result to (5/n-wideness. 

For j < n, we successively construct paths zi ■ ■ ■ Zj that are (5/2-' -wide and Zj 
is in the same region as and now is also ((5/27-separated. The state zi satisfies 
all requirements as zi is (5-separated. Let zi ■ ■ ■ Zj satisfy the requirements. In 
particular, Zj = (sj^qjjVj) is in the same region as z'j = {sj,qj,v'j). Since there 
is a waiting time t' with ^ z'^^-^ = (sj+i, gj+i, there is also an interval 

of waiting times (a, 6) such that for every t g (a, 6) we end up in the same 
region, i.e. Zj z for some z of the region containing z^+i. Moreover, due to 
(5/2-' -separation of lyj, we obtain 6 — a > . Therefore, we can choose the 
waiting time t = a + J • so that also lyj + t is 5/2''+-'^ -separated. Hence 
also i^j+i := {i^j + t)[{x I i'j_^_i{x) = 0} := 0] is 5/ 2^+ ^-separated. We set 
Zj+i := (sj+i,(ji+i,i/j+i). 

Notice, that this approach guarantees that the fractional parts of the just 
reset clock are "in the middle" between the surrounding clocks. That is why we 
needed exponential, i.e. 2", deminution of the separation. Nevertheless, due to 
^-separation, for every x,y G X there are at least n values between frac{v{x)) 
and frac{v{y)) such that even if all were fractional values of other clocks, the 
state would be (5/n-separated. Also note that as the path is only n steps long, 
there can be at most n different clocks set between any two clocks. Since we 
know their ordering in advance, these n different positions are sufficient. □ 

Now, we can finally start with the promised proofs. Lemma lC.Sl is a corollary 
of Lemmata ICTTI and [CT2] 
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Lemma IC.5I Let the region graph GmxA f>e strongly connected and let p be 
the period of GmxA- Let k S {0, . . . ,p—l}, S > and R G Vk be a region. Then 
there is n G N such that for every S -separated z\ G T^^^ there is a {S/n)-wide 
path zi ■ ■ ■ Zn such that Zn is (S / n) -separated and z„ e i?. 
Moreover, we can set n = I^I~"'^/6J - p. 

Proof. In the region graph we have a partition of vertices to sets Vb, . . . , Vp-i 
due to Lemma [3. 121 Let us fix a fc S {0, . . . ,p — 1}. We can define an aperiodic 
oriented graph (14, Ek) where {R, R') G Ek if there is a path from the region R 
to the region R' of length exactly p in the region graph Gm x a ■ 

Let us fix (5 > and a region i? G Vfe. Due to the strong connectedness and 
aperiodicity of (Vfe, E}^) we have by Lemma rC.lll in the graph (Vfe, E^) from any 
region R' G Vk a. path to R of length x = l^l-i/Gj > Ll^fcl*'" '^'='~V6J . 

Hence, in the graph GmxA: we have from R' to R a path of length n = x ■ p. 

For every zi G ^%^xA have a feasible path Z1Z2 ■ ■ ■ z'j with z,'j G R. We 
get the {S / n)-wide path by applying Lemma FC. 121 □ 

Lemma IC.9[ Let the region graph GmxA be strongly connected. Let S > 0. 
Let z be a S-separated state. Then there is n G N, .such that for any n' > n there 
is a .state z* such that the following holds. For any state zi almost equal to z 
there is a {S/n)-wide path z\ - ■ ■ z„' such that Zn' — z* . 
Moreover, we can set n = \ V\ ■ 

Proof. Let Z be the set of clocks that are not relevant in z. For each clock 
a; G Z there is a region R^ such that clock x is reset in region R^ (we make this 
assumption in Section l3.3|) . Let us fix a state zi almost equal to z. From the 
strong connectedness we get a feasible path ziZ2 - ■ ■ z'„ that for each x E Z visits 
the region R^. Furthermore, n<\V\- \Z\ < \V\ ■ \X\. From Lemma [012] we get 
a ((5/n)-wide path ziZ2 • ■ • Zn that also for each x € Z visits the region R^. 

Notice that for a fixed state z, control state s and time t there is a unique 
location q and valuation ly, hence a unique state z' — (s, q, v) such that z A z' . 

Let ti, . . . , tn-i be the waiting times and si, . . . , s„ the control states on the 
path zi, . . . , z„. For any zi almost equal to zi we can build using the same waiting 
times and control states a path zi • • • z„. It is easy to see that for two almost 
equal states z,z a, control state s and a time t > the states z', z' determined by 
s and t are also almost equal. Inductively, we get that Zi is almost equal to Zi for 
each 1 < i < n. Hence, the path zi • • • z„ is also ((5/n)-wide because 5-wideness 
is defined only with respect to relevant clocks. We show that z„ = z„(= z*). 

We need a parametrized version of almost equality. For a set of clocks 3^ and 
two states z — (s,q,v) and z — {s,q,v) we say that they are [V-equal if z ~ z 
and for each x G 3^ we have v{x) — v(x). The states zi and zi are <Yi-equal 
where Xi — X \ Z. Let Xi be a set of clocks and Zi and Zi be A^-equal states. 
For any t > Q and two states z^+i and z^+i such that Zi z^+i and Zi z^+i 
we have z^+i and z^+i are {Xi U 3^)-equal where y is the set of clocks reset in 
Zi. We get that z„ and A'-equal, i.e. z„ = z„. It holds because all clocks 

from Z are reset on the path zi ■ • • z„. 
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Now, for arbitrary n' > n, we can stretch the path to zi ■ ■ ■ Zn • ■ • z^' ■ We 
get Zn' — Zn' for any starting zi almost equal to zi because z„ — Zn- From 
same states we can obviously take the same steps to the same successor states. 
Furthermore, we can easily take ((5/n)-wide transitions by similar arguments as 
in the proof of Lemma IC.4I □ 

C.2.4 Proof of Lemma [C?fl 

Lemma IC.7I Let R he a region. For each 5 > Q there is (5' > 0, n G N and 
z' € r^v^x^ such that for every 5-separated zi G i? there is a 5' -wide path zi • • • z„ 
such that Zn and z almost equal. 

Moreover, we can set n = B^^^^y^ + 1 and 5' = (5/(i?,„ax + 2). 

Proof. No relevant clock has in zi its fractional value in the interval (0, 5) because 
zi is 5-separated. We divide this interval into -Bmax + 2 subintcrvals of equal 
length and set 5' — S/{B„jax + 2). 

For a fixed zi we inductively build a S'-wide path zi • • • z„ where n ~ i?,„ax+l. 
We fix an aribtrary linear order over the set of control states S of the semi- 
Markov process. Let 1 < i < n. For the state z^ = {si,qi,Ui) we choose as 
Si+i the first state (in the fixed order) such that P(si){si+i) > 0. This gives us 
a delay function / = 2)(si, Si+i). We set b to the integral upper bound of the 
interval where / is positive if it is not infinity. Otherwise, we set & = ^ -I- 1 where 
/ is the lower bound of /. Now, we fix the waiting time ti = b — S' and the state 
Zi+i {si+i,qi+i,i^i+i) such that Zi^Zi+i. 

We show that it is a i5'-wide transition. We divide the set of clocks into two 
disjunct subsets: the set of clocks y that have been reset in one of the states 
zi, . . . , Zi (have been reset at the beginning of the transition to the next state), 
and all other clocks y — X \ y. For each x G y lastly reset in state Zj where 
j < i we have frac{ui^i{x)) — 1 — {i + 1 — j) ■ S' , i.e. frac{vi^i{x)) < 1 — 6' 
and /rac(i-'i+i(x)) > 1 — S > S' . For each x d y we have frac{h'i^i{x)) = 
frac{vi{x)) -i-d' >S -i-d' >d'. Also, frac{iy,+i{x)) <l-d-i-S'<l-6'. 

We show that for any (5-separated starting state zi G i? we reach a state z„ 
almost equal to z„. We need a parametrized version of almost equality. For a 
set of clocks y and two states z = (s, q, v) and z = (s, q, u) we say that they are 
y-equal if z ~ z and for each x ^ y we have v{x) = v{x). The states zi and zi 
are 0-equal. Let Xi be a set of clocks and z^, Zi be A^-equal states. According to 
the inductive definition, we fix control states Sij^i,Si+i, waiting times i, and 
states Zi+i and z^+i such that Zi Zj+i and Zi -t- z^+i. Notice that s^+i — Si+i, 
hence t — t. We have Zj+i ^i+i. Furthermore, they are {Xi U 3^)-equal where 
y is the set of clocks reset in z^. We get that z„ and z„ are almost equal because 
the paths take at least i?max + 1 • (1 — 5') > i?max time units. All clocks not reset 
during this path become irrelevant. We finish the proof by setting z' = z„. □ 
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D Proofs of Section |4] 
D.l Proof of Theorem 14.11 



Theorem 14.11 For every i £N we have that 



( 



( 



Pmin ' 



) 



1 



c 



where c = 4 • \ V\. 

As 'Pa^(7?.) is equal to the probability of reaching C, the i-step transition 
probabilities Pj^^_^{z,C) converge to 'Pa^(7^) as i goes to infinity. Our goal is 
to show that they converge exponentially quickly. 

Our proof proceeds as follows. Denote by B the union of all regions that 
belong to BSCCs of GmxA- We show that for c = 4 ■ there is a lower bound 
Pbound > on the probability of reaching B in at most c steps from any state 
z € TmxA- Note that then the probability of not hitting B after i = m ■ c steps 
is at most (1 — Pbound)^- However, this means that P)^^j^{z,C) cannot differ 
from the probability of reaching C (and thus also from Vm{T^)) by more than 
(1 — Pbound)™ because C B and the probability of reaching C from B \ C is 
0. Moreover, we show that pbound can be set to (pmin • cjj • l/c)"^, from which we 
obtain the desired upper bound on \Vm(T^) — Pmxa{'^t^)\- 

So to obtain the desired result, it suffices to prove the following 

Proposition D.l. For every z G F^x^ we have that 



Here c^A-\V\ and pbound = {Pmin ■ c© • l/c)". 

Note that this section draws heavily on some of the methods and lemmas 
proved in the previous section, though often in a slightly easier form. However, to 
keep individual sections of the Appendix independent, we repeat the arguments 
here once more. 

Similarly to previous section, we are interested in paths z . . . z„ that are S- 
wide. For a fixed 5 > 0, it means that the waiting time of any transition in the 
path can be changed by ±S without ending up in a different region in the end. 
Precise definition follows. 

Definition D.2. Let z — (s, q, v) and z' — (s', q' , v') he two states. For a waiting 



For (5 > 0, we say that a feasible transition z ~> z' is 6-wide if for every x £ X 
relevant for v' we have frac{vi(x)) G [5, 1 — (5]. 

Let zi ■ ■ ■ Zn be a path. It is feasible if for each 1 < i < n we have that 
Zi — > Zi+i. It is (5- wide if for each 1 < i < n we have that Zi — >■ Zi+i is a 6-wide 
transition. 



^MxAi^^P) — Pbound 
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We first show that any (5-wide path of a finite length, say n, from any state 
z G r^x^ to a state Zn in a region R, induces a set of paths from z to the region 
R, and that their probability is bounded below by a positive constant. 

Lemma D.3. For every S > and n> 1 there is b > such that the following 
holds. For every 6 -wide path a = zqZi ■ ■ ■ Zn, there is a set of states Z ^ Zn such 
that it holds Pjl^^j^{zi, Z) > b. 

Moreover, we can set b = (pmin • od ■ 26 /n)'^. 

Proof. We fix any 5-wide path a = z^zi ■ ■ ■ Zn = 
(,so, (?o, J^o)('Si- 51. • ■ • For 1 < i < n, let ti be the waiting 

times such that z^, and let Xi — {x G X \ A{zi-i) = {q, v), v{x) = 0} be 

the set of clocks reset right before waiting ti. 

For e > 0, wc define an e-ncighbourhood of a to be the set of paths of the form 
Zo {si,qi,v[) ■ • ■ ^ [sn, Qn, Kl) whcrc t[ e {ti — e,t + e). Due to (5-wideness of 
(7, all paths of its (5/n- neighbour hood are feasible, and follow the same sequence 
of regions. Considering this 5/n- neighbourhood, the set of all possible f^s forms 
the sought set of states Z. 

We now give a lower bound on Pjl^y^j^iz, Z). First, recall the following nota- 
tion: let Pmin denote the smallest probability in M. Further, let us denote by 
D{M) the set of delay densities used in M, i.e. Ti{M) = {D(s, s') | s, s' e S}. 
Prom our assumptions imposed on delay densities we obtain the following uni- 
form bound cj) > on delay densities of S)(A^). For every / e ©(Al) and for 
all X G [0, -Bmax], either f{x) > c^ or f{x) — 0, and moreover, f{x)dx > c 

or equals 0. 

We define sets of states Zq, Zi . . . , Zn = Z, where Zi is the set of all states 
{sijqijvD in the ^-neighbourhood of a. Note that Pmxa{zoj ^i) = P(so)(si) • 

Ito-s/n ■l'd{t)dt> where fa is the appropriate delay density for this transition. 

Using the bounds given above, Pmxa{zo,Zi) > Pmin ■ It°^s/n = Pmin ■ cs • 
2S/n. Similarly, for any z^ G Zi, PMxA{z'i, Zi^i) > Pmin • • 2S/n holds by the 
same arguments. Therefore, from the definition of the n-step transition kernel, 

PJUxAi^O, Z) > (pmin ■ CD • 25/n)". □ 

We now prove that from any state z G TmxAj some BSCC reachable from z 
in the region graph is also reachable from z along a (5-wide path, and that this 
path length is bounded from above by a constant. 

We use two steps: first, we show that, from any z G Tj^xA^ we can reach 
a 5-separated state along J'-wide path of bounded length; second, once in a 6- 
separated state, we construct a 6" -wide path of length at most |y | ending in the 
BSCC. 

Definition D.4. Let 6 > 0. We say that a set X C M>o is 5-separated if for 
every x,y G X either frac{x) — frac{y) or \frac{x) — frac{y)\ > 5. 
Further, we say that {s, q, v) G TmxA is 5-separated if the set 

{0} U {y{x) \ x & X,x is relevant for u} 

is 5-separated. 
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Lemma D.5. There is S > and n G N such that for any z\ e ^ My.A there is 

a S-wide path zi • • • z„ such that 2„ is S-separated. 

Moreover, we can set n — B^i^^ ' (l-^l + 2) and 6 — 1/(2(|A'| + 2)). 

Proof. (Same as Lemma IC.4I) To simplify the argumentation we introduce a 
notion of a r-grid that marks r distinguished points (called lines) on the [0, 1] line 
segment. In the proof we show that we can place fractional values of all relevant 
clocks on such distinguished points. Let r G N. We say that a set of clocks 
y <^ X IS on r-grid in z if for every x & y relevant in z we have frac{i>{x)) = n/r 
for some < n < r. For < n < r, we say that the n-th line of the r-grid is free 
in z if there is no relevant clock in the l/2fc-neighborhood of the n-th line, i.e. 
for any relevant x G A" we have frac{v{x)) ^ (n/r — l/2r, n/r A- l/2r). 

Let r = |X| + 2. We inductively build a l/2r-wide path zi • • • z„ where 
n = -Bmax " f- The set is on r-grid in zi. We show that if a set is on r-grid in 
state Zi, there is a l/2fc-wide transition to z^+i such that U-E) is on r-grid in 
Zi+i where Z is the set of clocks newly reset in Zi. There are |X| -I- 2 lines on the 
grid and only \X\ clocks. At least two of these lines must be free. Let j ^ be 
such a line. Let i be a waiting time and z^+i a state such that frac{t) = 1 — j'/r 
and Zi Zi+i. Such waiting time must be indeed possible because the interval 
where the density function of any transition is positive has integral bounds. The 
transition z^+i is l/2r-wide because the line j is free in z^. Furthermore, 
the set {yi U Z) is on r-grid in z^+i because the fractional value of each clock 
that was previously on r-grid was changed by a multiple of 1/r. The newly reset 
clocks have fractional value 1 — j /r which is again a multiple of 1/r. 

Next, we show that X is on r-grid in z„. Clocks reset in this path on r-grid 
in z„. The remaining clocks are all irrelevant because the path of -Bmax • f steps 
takes at least -Bmax time units. Indeed, each transition in this path takes at least 
1/r time unit. According to the definition, X is on r-grid in z„. Hence, the state 
z„ is 1/r-separated because the distance between two adjacent grid lines is 1/r. 
By setting (5 = l/2r we get the result. □ 

Lemma D.6. Let S,d' > and R be a region. Then there is n G N such that 
for every S-separated z G TmkA it holds that if there is a feasible path from z to 
z\ for a z' in the region R, then there is also i < n and a S' -wide path z ■ ■ ■ Zi 
such that Zi G r_A/(x.A R is 5' -separated. 

Moreover, we can set n = \V\ and 5' — <^/|V^|. 

Proof. For simplicity, we first transform this path into a (5/2"- wide one. We then 
show how to improve the result to (5/n-wideness. 

Let us fix any (5-separated state z G F^x^, belonging to a particular region, 
say Rs- We will show that for any region R^ such that {Rx,Rs) & E in the 
region graph, we can find a waiting time t and ^-separated state zi belonging to 
Rx , such that z zi . 

As Rx is reachable from Rg in one step in the region graph, there is an 
interval of waiting times (a, b) such that for every t' G (a, b) z A- z[ for some z[ 
from Rx. Moreover, due to i5-separation of z, we obtain b — a > d. Therefore, we 
can choose the waiting time t — {a b)/2 and z[ is <5/2-separated. Intuitively, 
we need to 'lower' the ^-separation and wideness in each step as we might be 
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forced to reset a cfock, say x,., to a place between two other clocks, say Xi,X2, 
with \frac{xi — X2)\ — S. 

Note that if the state z' is reachable from z along a feasible path, it must be 
also reachable in at most |y| steps in the region graph. In such case, we can put 
n = \V\ and the 6' would be equal to (5/2". However, due to ^-separation, for 
every x,y G X there are at least n values between }rac{v{x)) and frac{v{y)) such 
that even if all were fractional values of other clocks, the state would be 5/n- 
separated. Also note that as the path is only n steps long, there can be at most 
n different clocks set between any two clocks. Since we know their ordering in 
advance, these n different positions are sufficient, and we can set 5' = 5 /\V\. □ 

Now we are ready to prove the Proposition lD.il 

of Proposition \D.1\ Lemma ID. 51 together with Lemma ID. 61 give us an upper 
bound on the number of steps needed to hit a state in one of the BSCCs along 
a (5-wide path from any state in TmxA'- we can set Cb — -Bmax • (l-^l + 2) + \V\ 
and 5 = (1/(2 • {\X\ + 2)). From Lemma ID31 we have 




As Cf, < 2 • for all but very small region graphs we have 




From this, we get the desired 





where c = 4 • 



□ 
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